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Sergeant  Roger  McCullough,  Major  Dennis  Ratashak,  Captain  Bradley  Wolfing,  and  Douglas  Dilday, 
Michael  Hayes,  and  John  Wasson.  “Tarantula”  observer-controllers  at  the  NTC  evaluated  task 
performance  by  the  48th  Brigade’s  Task  Force  2-121.  Joseph  Hagman  and  Monte  Smith,  Army  Research 
Institute  researchers,  provided  preliminary  tank  gunnery  performance  information  from  an  ongoing 


assessment  of  tank  gunnery  training  in  the  ARNG.  Chief  Warrant  Officer  Larry  Shue,  of  the  ARNG 
Readiness  Center,  furnished  Standard  Installation/Division  Personnel  System  (SIDPERS)  data  runs  that 
enabled  us  to  identify  numbers  of  personnel  targeted  for  training  by  the  various  SIMITAR  interventions  in 
the  test  brigades.  Anthea  DeVaughan,  Lee  Ann  Gehman  (nee  Miller),  and  Susan  Taylor  of  IDA  typed  the 
report.  John  (Chuck)  Everett  did  the  editing.  Numerous  officers  and  staff  noncommissioned  officers  of 
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III-5  Summary  of  Performance  at  NTC . Ill- 13 
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III-l.  r  Distributions  for  Two  Example  Comparisons .  ni-6 
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units;  however,  since  no  ARNG  units  had  recently  trained  at  the  NTC,  there  was  no  such  benchmark  for 
the  48th  Brigade  at  NTC  96. 


The  fact  that  some  of  the  518  tasks  are  trainable  by  SIMITAR  devices  and  courses  while  other  tasks 
are  not  gives  us  an  alternative  benchmarking  option:  performance  on  tasks  not  trained  by  SIMITAR 
benchmarks  the  performance  on  tasks  trained  by  SIMITAR. 

48th  Brigade.  Results  from  our  assessment  of  the  48th  Brigade’s  field  exercises  at  NTC  96  (see 
Tables  ni-5  and  III-6)  follow: 
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and  IV-S). 

Different  Focuses.  The  NTC  training  evaluation  focuses  on  the  large  collectives  at  higher 
organization  levels  (brigade,  battalion)  whereas  most  SIMITAR  devices  and  courses  train  individuals  and 


small  collectives  (crew,  squad,  platoon).  Additional  observer-controllers  (OCs)  are  needed  for  the  116th 
Brigade  at  NTC  98  (than  were  involved  at  NTC  96)  if  we  are  to  evaluate  SIMITAR  devices  and  courses 
other  than  Janus,  SIMNET,  Advanced  Research  Projects  Agency  Reconfigurable  Simulator  Initiative 
(ARSI),  and  perhaps  Battle  Staff  Training  System  (BSTS)  and  Combat  Service  Support  Interactive 
Courseware  (CSS  ICW).  AT  97  presents  an  opportunity  to  assess  task  performance  in  the  field  by  116th 
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I.  INTRODUCTION 
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Our  task  objective  is  to  evaluate  the  effectiveness  of  SIMITAR  technology  to  improve  the 
performance  of  ARNG  brigades  in  exercises  at  the  National  Training  Center  (NTC)  and  in  other  training- 
related  measures. 
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Visual  Reality  Maintenance  Trainer  (VMAT). 

Training  devices  for  small  unit  and  collective  training  include: 

Abrams  Full-Crew  Interactive  Simulator  (AFIST) 

Advanced  Research  Projects  Agency  Reconfigurable  Simulator  Initiative  (ARSI) 
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Janus  for  a  brigade  staff  and  Janus  for  a  battalion  staff  are  considered  different  interventions. 


Two  interventions.  Compressed  Gunnery  and  Pile-on  Training,  are  strategies  to  achieve  training 
synergism.  Compressed  Gunnery  mixes  live  fire  and  simulation  throughout  the  year  to  maximize  use  of 
available  training  time.  Pile-on  weekends  involve  integrated  usage  of  gunnery  simulators  (COFT,  AFTST) 
and  maneuver  simulators  (ARSI,  SIMNET)  to  use  inactive  duty  training  (IDT)  periods  more  effectively. 

The  SIMBART  and  SIMUTA  trainers  are  structured  exercises  that  provide  training  in  brigade  and 


<U 

a 

)  <D 
c/5 


)  5 


.s  > 

G  ^ 
G  ^ 
G  G 

■H-  8 

2^  G 
O  G 


G  O 
O 
c/5 

TD 

c/5  ^ 

o  5 


■S! 

O  ’H< 

^  -a  a 
3  o 
<u  g 

pa. 

•2  < 
.  -a  H 
^  o  D 

•G  0)  ^ 
w  c/5 

^  S  f-I 

C2U  ^  .5 

1/5 

2i  ti  — 
,  ’3 

gj  gj  o 

O  0^ 


C  3>  o 

§  .g  ^ 

o  *-  S 

i2  o  .S 
W5  a  e 

■3  ^  is 

cd  0) 

x>  cu  £ 

.2  «  g 

2  I  -3 


ojo 

G  X 
G  -2 

a  I 
§ 

««  ^ 
o  c 
42  .o 

c/5 

G 

CO  S 

0)  •G 
•  SS 

lg 

«  z 

>  CO 
G 

<U  ^ 
G  (U 
G  > 

S 


^  c/5 

o  G 

=3  >a 
"O  3  (u 

§  60  S 

S  ‘C  «« 

ffl  "" 
60  <U 
C  _e- 

•=  ’S  ■“ 

•  S  ^  C 

cd  ^ 

'o 

s  =* 

5  ^ 
2  Z 
§  oo  2 

I  ^  < 

D  i-t 

I  -  3 

S'*-® 
,a  ®  ^ 

CO  O 

6  a  I 

o  a  S 

o  e  ^ 

60 't:  J3 

E  «  2 

'S  ^ 

I  '§  § 

2  a 
•r  ^  2 

53  §  i 

rv  O  ^ 

s  S  -S 

8  -S  I 

Cfl  <4-1  O 
«  O  O 

Z  o 
0  3*^ 

.S  •S 

ca;  tr  -S 

a  3  52 
§  ®  3 

CO  S 

•fa  s 

I  S'! 

>  =  85 

w  ■!< 


6  H 

G  cd 

^  D 
(U  Uh 

G  G 

CO 

G  Cd 

::  I 

^  r- 

G 

G  ^ 

I H 

in 

2  u 
ci« 

2 
o  3 


®  1 
G 

CO 

(U  ^ 

CO  ^ 

’o  ^ 

0)  c 
X  o 

«  3 
—  a 


.2  g 

2  i 

a  s 

S  M 

G  c 

-G 
o  c 


G  S 
> 

00  (U 

■il 

1-1  bO 
^  G 
-22 
O  ^G 

§  ^ 
pG 

p<  G 

(U  <u 

a. 


^  CO 

S  Ji 


cP  ^ 


.G  ^ 

4-»  *2 

G  g 

S  G 
’S  3 

^  g 

8  2 

8  « 
CO 

00  T3 
G  G 

.S  o 
g  »-• 

'4-H 

H  O 


o  ^ 

^  G 
o 

o'  o 
X  s 

rj  1-^ 

.5  o 
ex 
G  a. 
o  o 

‘•G  w 
G  G 
O  <u 

€  g 

G  CO 

G  D 

cr  «« 

<D  ^ 

C  f_i 

o  a  U 
CO  Z 
2  -g  -s 

o  ^  G 

O  T3  G 

^  S  ^ 
a  i  § 
>  2  r< 


<U  5.4 

O  < 


w  b  c 

i  "t: 

KJ  (D  Ui 

H  ca  H 


1 


While  we  have  collected  substantial  data  on  the  amount  of  training  conducted  on  various  SIMITAR 
devices  at  home  station,  we  are  unable  to  find  data,  except  for  tank  gunnery,  on  corresponding  training 
performance.  We  ultimately  want  to  compare  training  accomplishment  and  achievement  in  the  SIMITAR 


brigades,  which  are  Enhanced  Readiness  Brigades  (ERBs),  with  training  accomplishment  and  achievement 
in  other  ERBs  that  do  not  have  SIMITAR. 
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A  standard  for  commanders  to  assess  training  performance  of  Reserve  Component  units.  Guidance  to  evaluator  personnel 
and  instructions  for  preparing  TAM  Form  1049-R  are  provided  in  FORSCOM  Regulation  220-3. 


brigades.  Chapter  HI  describes  the  framework  for  assessing  SIMITAR  at  NTC  96,  and  it  gives  a  statistical 
description  of  the  training  performance  of  the  48th  Brigade.  Chapter  IV  describes  training  trends  in  tank 
battalions  of  the  116th  Brigade  during  the  period  in  which  SIMITAR  has  become  a  major  factor  in  their 
training  program.  Chapter  V  provides  assessments  of  SIMITAR  interventions  by  116th  Brigade  officers 
and  RTD  officers  who  have  recently  transferred  or  retired.  Chapter  VI  describes  work  to  be  done. 


SO 
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Table  M.  SIMITAR  Interventions 
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Table  1-2.  SIMITAR  Interventions:  Functional  Descriptions 

AFIST  Improvements  to  an  existing  low-cost,  transportable  multimedia  tank  gunnery  simuiator  that 

_ is  attached  to  a  stationary  tank  located  in  a  training  area. _ 

A  R  S I  Reconfigurable  simulation  piatform  capable  of  emulating  Abrams  and  Bradiey  armored 

vehicles  and  High  Mobiiity  Maneuverabie  Wheeled  Vehicles  (HMMWVs). 
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II.  SIMITAR  USAGE 
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Source:  SIDPERS  (Standard  Installation/Division  Personnel  System),  1  January  1997  data  run  showing  assigned  personnel 
who  are  MOS  qualified. 

If  a  staff  noncommissioned  officer  (NCO)  with  the  specified  skill  level  is  not  available  for  Janus  battle  staff  exercises,  we 
assume  the  next  lower  skill  level  with  the  same  MOS  will  participate. 


assigned  personnel  who  are  not  MOSQ  in  the  48th  Brigade  and  the  116th  Brigade,  respectively,  are 
undergoing  or  awaiting  basic  training  and  MOS  training. 
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Appendix  B  shows  the  computation  of  cumulative  training  hours  available  and  SIMITAR  training  hours. 


The  following  rules  and  comment  apply  to  counting  training  hours  for  the  SIMITAR  interventions: 

•  Training  time  includes  briefings  or  preparatory  activity  immediately  before  the  training 
exercise  and  post-exercise  feedback  discussions. 

•  Any  training  on  COFT,  SIMNET,  or  any  other  SIMITAR  intervention  away  from  home  station 
is  counted. 
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usage  by  TF  rather  than  battalion  in  training-hour  accounting.  Table  II-6  shows  training  man-hours  in 
battle  staff  synchronization  (Janus),  maneuver  (SIMNET  and  ARSI),  and  gunnery  (COFT  and  AFIST)  for 


TFs.  Appendix  C  shows  the  battalion-to-task  force  transformation  and  the  training  hour 
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‘Battle  staff  training”  means  training  in  battle  staff  synchronization  or  battlefield  synchronization. 


Table  II-9  shows  that  over  75  percent  of  assigned  personnel  in  the  2-116  and  3-116  battalions  are 
MOSQ  and  over  80  percent  of  the  latter  receive  training  by  one  or  more  SIMITAR  interventions.  Table 
II-8  also  shows  that  SIMITAR  training  in  gunnery,  maneuver,  and  battle  staff  account  for  about  20  percent 
of  total  IDT  training  hours;  gunnery  training  in  tanks  and  maneuver  training  in  the  field  account  for 
another  20  percent.  Because  of  increasing  usage  of  simulators  by  the  116th  Brigade  tank  battalions  in 
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Table  11-1.  MOSs  and  Duty  Positions  Trained  by  SIMITAR  Interventions,  as  of  November  1996 


TT 


Table  11-2.  Number  of  Personnel  Who  Are  Training  Targets  for  SIMITAR  Interventions 
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Number  of  personnel  who  are  MOS  qualified. 


Table  il-3.  SIMITAR  Training  in  the  Two  Test  Brigades 


Table  11-4.  Date  of  First  Use  of  SIMITAR  Interventions 
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Table  11-5.  Man-hours  Spent  Training  on  SiMITAR  interventions  From  Start  to  NTC  96 
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Table  11-6.  Training  Man-hours,  Through  Aprii  1996 
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Table  ll-?.  Man-hours  of  Home  Station  Training  in  the  2-116  Battalion 
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Table  11-8.  Man-Hours  of  Home  Station  Training  in  the  3-116  Battaiion 
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The  BOSs  are  functions  that  help  commanders  build  and  sustain  combat  power. 

Within  the  BOSs,  CCFs  are  a  set  of  collective  activities  that  units  must  perform  successfully  to  survive  and  win  battles. 


The  CCFs,  which  were  developed  by  the  Army  Research  Institute  (ARI),  incorporate  information 
from  the  Army  Training  and  Evaluation  Program  (ARTEP)  Mission  Training  Plans  (MTPs)  and 
information  from  other  sources  of  task  documentation,  such  as  Field  and  Technical  Manuals. 

We  directed  our  evaluation  at  14  CCFs  in  which  ARNG  units  were  most  deficient  in  past 
performance  at  the  NTC.  Table  III-l  identifies  these  14  CCFs.  SIMITAR  interventions  train  at  the  task 
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staff  was  less  familiar  with  their  capabilities. 
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and  Untrained  (U).  These  ratings  were  made  without  the  knowledge  of  which  tasks  were  trainable  or  not 
trainable  by  SIMITAR  interventions.  Later,  these  assessments  were  coded  on  a  simple  linear  scale,  where 
T  =  2,  P  =  1,  and  U  =  0. 


Table  III-4  illustrates  how  the  assessment  data  were  coded  for  analysis.  This  table  shows  only  a 
portion  of  the  beginning  and  end  of  the  matrix.  Note  that  some  cells  in  the  matrix  have  no  entries. 
Missing  entries  indicate  that  the  unit  did  not  perform  the  corresponding  task  or  that  the  assessment  team 
was  not  able  to  observe  performance  of  that  task. 

Table  III-5  summarizes  average  performance  scores  by  BOS  for  the  four  observed  units:  brigade. 


C/3 


<U 


i  - 


C/3 


CO 


4-^  I 

^  cd  <U 

c/5 

o  e  c3 
0)  O  ^ 

^  U  M 
or  ^  Q 

^  o  y 


(U 


°  ^  a 


D 


c/5 

d) 

•  ^ 
Im 

was 

d) 

’S 

u 

H 

O 

Z 

cxo 

c/5 

G 

C/5 

d) 

o 

§ 

o 

cd 

o 

o 

Uh 

w 

G 

o 

04 

W) 

G 

1-H 

2 

cx. 

ta 

cd 

> 

o 

.c 

u 

G 

C 

O 

U 

-w 

C/5 

c/5 

s 

O 

J2 

C/5 

O 

s 

<U 


<U 

c/5 

c 

o 

d) 


T3  O 

a  ^ 

cd  c/j 

<U 

c/5 


C/5 


cd  cd 


dD 

C 


Cd 

H 

X 

< 

H 

s 

C/5 

I 

c 

o 

2 

CA 

9 

4) 

> 

C 


9 

U 

H 

H 


CO 


(D 


d>  O 

'*-»  c/5 


I 


determine  whether  the  differences  in  average  performance  on  SIMITAR  trainable  vs.  SIMITAR  non- 
trainable  tasks  were  significant  (i.e.,  not  due  to  chance).  We  tested  directional  hypotheses  (i.e.,  one-tailed 
tests)  using  the  following  formula: 
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Table  III-6  shows  that  the  48th  Brigade  HQ  staff,  which  had  very  little  Janus  training,  performed 
slightly  worse  on  SIMITAR  trainable  tasks  than  on  other  tasks.  In  evaluating  the  performance  of  the  two 
TFs,  we  recall  the  training-hour  data  from  Table  II-6. 
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false  positive  decision  (declaring  a  difference  significant  when  there  is  no  actual  difference).  In  short,  the 
effect  of  these  sources  of  variability  was  to  make  the  analyses  more  conservative  but  not  invalid. 


A  second,  and  perhaps  more  serious,  threat  to  the  validity  of  these  analyses  is  variables  that  are 
correlated  or  confounded  with  differences  between  SIMITAR  trainable  and  other  tasks.  Perhaps  the  most 
obvious  confound  is  the  possibility  that  SIMITAR  trainable  tasks  are  inherently  easier  than  other  tasks. 
This  could  explain  why  task  performance  of  TF  2-121  was  better  on  SIMITAR  trainable  tasks  than  on 
SIMITAR  non-trainable  tasks.  However,  the  results  from  TF  1-108  and  the  brigade  staff  do  not  support 
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Table  III-2.  Portion  of  Task  vs.  Intervention  Matrix 


in- 10 


Table  III-3.  Number  (Percent)  of  Tasks  Trained  by  Interventions 
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Does  not  train  brigade-  and  battalion-level  tasks  in  Fire  Support;  trains  only  one  task  (develop  Fire  Support  plan)  at  lower  echelons. 


Table  III-4.  Portion  of  Performance  Data  Matrix 
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Table  III-5.  Summary  of  Performance  at  NTC 
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Table  III-6.  Performance  on  Trainable  vs.  Not 
Trainable  Tasks  on  SIMITAR  Interventions 
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caused  by  (1)  more  complex  tasks  being  performed  and  evaluated  and  (2)  tougher  assessment  standards 
being  used.  Review  of  the  FY92-FY96  TAM  reports  confirms  that  maneuver  and  battle  staff  training  at 
AT  has  progressed  from  platoon- level  lanes  to  company — and  battalion-level — lanes,  where  tasks  are 


indeed  more  complex.  Discussion  with  an  active  Army  evaluator  also  confirmed  that  tougher  standards 
were  used  in  latter  AT  periods. 
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FY96,  battle  staff  C2  was  concerned  with  battalion-level  exercises,  including  CSS. 

While  Table  IV-8  shows  improvement  in  gunnery,  maneuver,  and  battle  staff  performance  for  3- 
1 16,  an  Oregon  battalion,  the  AT  results  are  not  as  impressive  as  those  for  2-116,  an  Idaho  battalion,  in  the 


preceding  table.  The  differences  in  AT  performance  of  the  two  battalions  appears  attributable  to  different 
training  circumstances,  which  favor  2-116: 

•  AFTST  was  available  in  2- 1 1 6  over  a  year  before  3-116  got  AFIST. 

•  While  2-116  has  four  AFISTs  to  train  its  four  companies,  3-116  has  five  companies  plus  the 
116th  Brigade  Cavalry  Troop  to  train  on  its  four  AFISTs;  one  of  the  tank  companies  was 
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FYs  1993  through  1996  shows  that  116th  Brigade  tank  crews  had  36  percent  first-run  qualification  (Ql) 
and  53  percent  second-run  qualification  (Q2)  for  a  total  of  89  percent  Ql  or  Q2;  the  comparable 
performance  for  other  ERB’s  tank  crews  was  53  percent  Ql  and  35  percent  Q2  for  a  total  of  88  percent 
Ql  or  Q2. 


Several  officers  and  staff  NCOs  in  SIMITAR  tank  units  and  in  their  Resident  Training  Detachments 
attribute  the  relative  decline  of  the  1 16th  Brigade  in  TT  VIII  qualification  to  less  time  in  tanks  and  more 
time  in  gunnery  simulators.  Increased  dependence  on  simulators  for  home  station  training  is  of  course  the 
essence  of  the  SIMITAR  training  program,  which  trains  both  gunnery  and  maneuver  every  year  whereas 
other  ERBs  reportedly  focus  on  gunnery  training  one  year  and  maneuver  training  the  next.  An 
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Table  IV-1.  Five  Years  of  TAM  Evaluations  for  the  2*116  Battalion 
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Table  IV-2.  Five  Years  of  TAM  Evaluations  for  the  3-116  Battalion 
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Table  IV-3.  TAM  Scores  Derived  From  Commander’s  Assessments  for  the  2-116  Battalion 
[Trained  (T)  =  2;  Needs  Practice  (P)  =  1;  Untrained  (U)  =  0] 
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Table  IV-4.  TAM  Scores  Derived  From  Commander’s  Assessments  for  the  3-116  Battalion 
[Trained  (T)  s  2;  Needs  Practice  (P)  =  1;  Untrained  (U)  s  O] 
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Table  IV-6.  TAM  Scores  Derived  From  Evaluators’  Assessments  for  the  3-116  Battalion 
[Trained  (T)  =  2;  Needs  Practice  (P)  s  1;  Untrained  (U)  s  0] 
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Table  IV-9.  Proportion  of  Tank  Crews  in  Selected  ERBs  Qualifying  in 
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Table  V<2.  Comments  From  116th  Brigade 
About  SIMITAR  Interventions 
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Table  V-2.  Comments  From  116th  Brigade 
About  SIMITAR  Interventions  (Continued) 
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Table  V-2.  Comments  From  116th  Brigade 
About  SIMITAR  Interventions  (Continued) 
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Increases  the  ability  of  a  unit  to  train  up  to  100  percent 
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Table  V-2.  Comments  From  116th  Brigade 
About  SIMITAR  Interventions  (Continued) 
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Table  V>2.  Comments  From  116th  Brigade 
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Table  A-1.  Number  of  Personnel  Who  Can  Be  Trained  by  19  SIMITAR  Interventions,  by  MOS 

and  Duty  Position  (Continued) 
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Table  B-1.  Available  Man-Hours  for  Training  by  SIMITAR  Interventions  in  the  48th  In  Bde 
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Table  B-2.  Available  Man-Hours  for  Training  by  SIMITAR  Interventions  in  the  116th  Cav  Bde 
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Table  C-2.  Transformation  of  SIMITAR  Usage  by  Battalions  at  Home  Station  to  Task  Forces 
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Table  D-1.  Thirty-nine  Critical  Combat  Functions  in  Seven  Battlefield  Operating  Systems 

INTELLIGENCE  (19)  Direct  and  Lead  Unit  During  Preparation 

(1)  Conduct  Intelligence  Planning _  (20)  Direct  and  Lead  Unit  in  Execution _ 

(2)  Collect  Information _  MOBILITY  &  SURVIVABILITY _ 

(3)  Process  Information  (21)  Overcome  Obstacles 
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Table  E-1.  Mean  Performance  Score  at  the  NTC  Per  Task  by  BOS,  With  Frequency  of  Ratings 
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Table  E-1.  Mean  Performance  Score  at  the  NTC  Per  Task  by  BOS,  With  Frequency  of  Ratings 

(Continued) 

_ Frequency  (Percent)  of  Ratings _ 

Battlefield  Operating  No.  of  Trained  Needs  Untrained  Mean 

System  Ratings  Practice  (P)  /ii\  Score 
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Compressed  Gunnery 

.  Describe  the  impact  that  Compressed  Gunnery  has  had  on  your  unit. 

a.  Give  statistical  evidence  of  performance  improvements,  if  any,  in  Tank  Gunnery  Tables  VIII 
and  XII. 
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.  (Be  as  specific  as  possible.) 


Digital  Systems  Test  and  Training  Simulator  (DSTATS) 
Describe  the  impact  that  the  DSTATS  has  had  on  your  unit. 

a.  How  many  times  has  your  unit  used  DSTATS?  (Provide  dates  if  possible.) 
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Engagement  Skills  Trainer  (EST) 


F-14 


Forward  Support  Battalion  (FSB)  Training  Packages 
.  Describe  the  impact  that  FSB  Training  Packages  has  had  on  your  unit. 

a.  How  has  your  unit  used  FSB  Battle  Staff  training  courseware?  FSB  Lanes? 
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Guard  Unit  Armory  Device,  Full-Crew  Interactive  Simulation  Trainer,  Field  Artillery 

(GUARDFIST  II) 


III 


T 


F-16 


.  Describe  the  impact  Janus  technology  has  had  on  your  unit. 
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.  Describe  the  impact  that  Mobile  SIMNET  has  had  on  your  unit. 

a.  How  many  times  has  your  unit  used  Mobile  SIMNET?  (Provide  dates  if  possible.) 
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Pen-Based  Electronic  Network  for  Command  Information  Linking  (PENCIL) 
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Describe  the  impact  that  Pile-On  IDTs  has  had  on  your  unit. 

a.  How  many  times  has  your  unit  participated  in  a  Pile-On  IDT?  (Provide  dates  if  possible.) 
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Reserve  Component  Virtual  Training  Program  (RCVTP) 
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.  Describe  the  impact  that  the  S-2  Trainer  has  had  on  your  unit. 
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Simulation-Based  Mounted  Brigade  Training  Program  (SIMBART) 
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